Over the recent years cellular phones, digital cameras and MP3 players have collectively outsold desktop or laptop computers by almost an order of magnitude. In some parts of the world the cellular phone is the only computing device possessed by individuals. Many phones today are multifunction devices and include digital cameras, music and video playback, several gigabytes of storage either in flash or small drives, and also short range wireless capabilities. While some include just a numeric keypad, others include full function keyboards or touch panels. In the near future mobile devices are also expected to include near field communication (NFC) tag readers and also incorporate Trusted Platform Modules (TPM). As a result mobile devices are a hotbed for deploying a wide range of services ranging from several forms of communications, web browsing and transactions to geo-tagged photography and geo-blogging. While mobile devices have the distinct advantage in number and ubiquity, they are, by their very nature, limited in size, and therefore pose a usability challenge when acting as a computing interface.
In a service oriented architecture that is deployed in public spaces, heterogeneity of services and devices is unavoidable. Late binding of the mobile device's user interface for interacting with such a service becomes important for several reasons. A device that uses the service may not have been built when the service was conceived and so the service may not know how to provide a suitable user interface on the device. In other cases, the device may not have been popular when the service was created and therefore not drawn to the attention of the service developer. In particular, a service author would not have been expected to predict the set of devices that will access the service, and a device manufacturer cannot preload interfaces for all types of services. Additionally, over the course of the deployment of the service, specific details of the service may change, thereby requiring a refinement of the interface.
There currently does not exist any efficient technology that will enable associations between services and devices for supporting effective interaction of users with services in their environment via their personal mobile devices to be created automatically without any custom programming.
In general, automatically creating such associations is difficult for a variety of reasons. First, there is a wide diversity in the types of input controls that are available on mobile devices. Some devices such as mobile phones support a set of push buttons, while other devices such as PDAs may include touch sensitive screen, devices such as Apple's iPod have a wheel that can be used to scroll through large amounts of content at variable speeds. Also the relative sizes and placements of the input controls on a particular device may be convenient to operate them in a one-handed or eyes-free manner while other devices may require a far greater level of user attention. Some devices may also support voice recognition and be able to convert voice commands into simple events that they can deliver to remote services. Given the advances in display technology, such as flexible organic LED displays, over time, it is expected that physical buttons and controls to include small displays allowing user's to change the labels on the physical controls. Such a development could reduce the reliance on a GUI on a mobile device and allow better use of the limited screen space on the device for output rather than for input widgets. As a result of the above, the set of input events that can be generated from a mobile device may be extremely varied depending on its capabilities.
Secondly there is a wide diversity in the types of environmental devices and applications or services in the environment, and each of these may support a large collection of control operations. A simple device such as a thermostat may support a few operations such setting the desired temperature and querying the current temperature. A more complex service such as purchasing a roundtrip ticket for a train journey requires a larger set of control operations.
Third, users may have their own individual preferences in terms of which control manipulations are easier for them to perform. Some users may prefer to use a rotary jog dial for scrolling operations while others may like specific push buttons to scroll up or down.
Thus, it would be highly desirable to provide a system and method that addresses the need for mobile and pervasive digital devices to generate user interfaces that take into account the user's dynamic preferences, the device's characteristics and the service's characteristics.
Moreover, it is desirable to provide a mechanism that maps a user device's interface for the service to the input controls of the device when the service is invoked. In contrast to previous models where functions are mapped to I/O devices at application creation time or application installation time, or pick a predefined mapping based on device type, it would be much more efficient and highly desirable to provide a system and method for automatically mapping the interface at service invocation time by taking into account, device characteristics, user preferences, and service characteristics.